Deep Learning from the Basics (2021) [Saitoh] [9781800206137]

Cover
FM
Copyright
Table of Contents
Preface
Introduction
Chapter 1: Introduction to Python
	What is Python?
	Installing Python
		Python Versions
		External Libraries That We Use
		Anaconda Distribution
	Python Interpreter
		Mathematical Operations
		Data Types
		Variables
		Lists
		Dictionaries
		Boolean
		if Statements
		for Statements
		Functions
	Python Script Files
		Saving in a File
		Classes
	NumPy
		Importing NumPy
		Creating a NumPy Array
		Mathematical Operations in NumPy
		N-Dimensional NumPy Arrays
		Broadcasting
		Accessing Elements
	Matplotlib
		Drawing a Simple Graph
		Features of pyplot
		Displaying Images
	Summary
Chapter 2: Perceptrons
	What Is a Perceptron?
	Simple Logic Circuits
		AND Gate
		NAND and OR gates
	Implementing Perceptrons
		Easy Implementation
		Introducing Weights and Bias
		Implementation with Weights and Bias
	Limitations of Perceptrons
		XOR Gate
		Linear and Nonlinear
	Multilayer Perceptrons
		Combining the Existing Gates
		Implementing an XOR Gate
	From NAND to a Computer
	Summary
Chapter 3: Neural Networks
	From Perceptrons to Neural Networks
		Neural Network Example
		Reviewing the Perceptron
		Introducing an Activation Function
	Activation Function
		Sigmoid Function
		Implementing a Step Function
		Step Function Graph
		Implementing a Sigmoid Function
		Comparing the Sigmoid Function and the Step Function
		Nonlinear Function
		ReLU Function
	Calculating Multidimensional Arrays
		Multidimensional Arrays
		Matrix Multiplication
		Matrix Multiplication in a Neural Network
	Implementing a Three-Layer Neural Network
		Examining the Symbols
		Implementing Signal Transmission in Each Layer
		Implementation Summary
	Designing the Output Layer
	Identity Function and Softmax Function
		Issues when Implementing the Softmax Function
		Characteristics of the Softmax Function
		Number of Neurons in the Output Layer
	Handwritten Digit Recognition
		MNIST Dataset
		Inference for Neural Network
		Batch Processing
	Summary
Chapter 4: Neural Network Training
	Learning from Data
		Data-Driven
		Training Data and Test Data
	Loss Function
		Sum of Squared Errors
		Cross-Entropy Error
		Mini-Batch Learning
		Implementing Cross-Entropy Error (Using Batches)
		Why Do We Configure a Loss Function?
	Numerical Differentiation
		Derivative
		Examples of Numerical Differentiation
		Partial Derivative
	Gradient
		Gradient Method
		Gradients for a Neural Network
	Implementing a Training Algorithm
		A Two-Layer Neural Network as a Class
		Implementing Mini-Batch Training
		Using Test Data for Evaluation
	Summary
Chapter 5: Backpropagation
	Computational Graphs
		Using Computational Graphs to Solve Problems
		Local Calculation
		Why Do We Use Computational Graphs?
	Chain Rule
		Backward Propagation in a Computational Graph
		What Is the Chain Rule?
		The Chain Rule and Computational Graphs
	Backward Propagation
		Backward Propagation in an Addition Node
		Backward Propagation in a Multiplication Node
		Apples Example
	Implementing a Simple Layer
		Implementing a Multiplication Layer
		Implementing an Addition Layer
	Implementing the Activation Function Layer
		ReLU Layer
		Sigmoid Layer
	Implementing the Affine and Softmax Layers
		Affine Layer
		Batch-Based Affine Layer
		Softmax-with-Loss Layer
	Implementing Backpropagation
		Overall View of Neural Network Training
		Presupposition
		Implementing a Neural Network That Supports Backpropagation
		Gradient Check-in Backpropagation
		Training Using Backpropagation
	Summary
Chapter 6: Training Techniques
	Updating Parameters
		Story of an Adventurer
		SGD
		Disadvantage of SGD
		Momentum
		AdaGrad
		Adam
		Which Update Technique Should We Use?
		Using the MNIST Dataset to Compare the Update Techniques
	Initial Weight Values
		How About Setting the Initial Weight Values to 0?
		Distribution of Activations in the Hidden Layers
		Initial Weight Values for ReLU
		Using the MNIST Dataset to Compare the Weight Initializers
		Batch Normalization
		Batch Normalization Algorithm
		Evaluating Batch Normalization
	Regularization
		Overfitting
		Weight Decay
		Dropout
	Validating Hyperparameters
		Validation Data
		Optimizing Hyperparameters
		Implementing Hyperparameter Optimization
	Summary
Chapter 7: Convolutional Neural Networks
	Overall Architecture
	The Convolution Layer
		Issues with the Fully Connected Layer
		Convolution Operations
		Padding
		Stride
		Performing a Convolution Operation on Three-Dimensional Data
		Thinking in Blocks
		Batch Processing
	The Pooling Layer
		Characteristics of a Pooling Layer
	Implementing the Convolution and Pooling Layers
		Four-Dimensional Arrays
		Expansion by im2col
		Implementing a Convolution Layer
		Implementing a Pooling Layer
	Implementing a CNN
	Visualizing a CNN
		Visualizing the Weight of the First Layer
		Using a Hierarchical Structure to Extract Information
	Typical CNNs
		LeNet
		AlexNet
	Summary
Chapter 8: Deep Learning
	Making a Network Deeper
		Deeper Networks
		Improving Recognition Accuracy
		Motivation for a Deeper Network
	A Brief History of Deep Learning
		ImageNet
		VGG
		GoogLeNet
		ResNet
	Accelerating Deep Learning
		Challenges to Overcome
		Using GPUs for Acceleration
		Distributed Training
		Reducing the Bit Number for Arithmetic Precision
	Practical Uses of Deep Learning
		Object Detection
		Segmentation
		Generating Image Captions
	The Future of Deep Learning
		Converting Image Styles
		Generating Images
		Automated Driving
		Deep Q-Networks (Reinforcement Learning)
	Summary
Appendix A
Index